The present invention relates to a method for detecting flicker noise, and more particularly, to a method for detecting flicker noise produced when generating an image with an imaging device, such as a digital still camera or a digital video camera, indoors near a fluorescent light.
A fluorescent light may be used as a lighting device in a room. In such a room, when an imaging device, such as a digital still camera or a digital video camera, is used to generate an image, the generated image may include flicker noise in accordance with the relationship between the exposure time of photoelectric conversion elements, which configure an image sensor, and the fluorescence frequency of the fluorescent light. In Japan, the AC power frequency differs between regions. This results in fluorescent lights having different fluorescence frequencies. Thus, flicker noise is produced under different conditions. Under such circumstances, it is required that the detection of flicker noise be ensured and performed quickly to reduce flicker noise.
In regions of Japan where the AC power frequency is 50 Hz, the fluorescence frequency of a fluorescent light is 100 Hz. In the other regions of Japan where the AC power frequency is 60 Hz, the fluorescence frequency of a fluorescent light is 120 Hz. An auto gain control (AGC) circuit is incorporated in a digital still camera or a digital video camera. The brightness of an imaging subject ranges from a high level to a low level. Thus, the AGC circuit automatically adjusts the frame rate in accordance with the brightness of the imaging subject.
In an XY address type CMOS image sensor, photoelectric conversion elements are respectively connected to horizontal selection lines. The total of the vertical scanning period and the vertical blanking period from the first horizontal selection line to the final horizontal selection line corresponds to a cycle of one frame.
For example, when the image sensor is operating at a frame rate of 30 fps, a cycle is 1/30 seconds per frame. In this cycle, each horizontal selection line is provided with a reset signal and a read signal. The time from when the reset signal is provided to when the read signal is provided corresponds to the exposure time (integration time or signal storage time) of each photoelectric conversion element.
When the cycle of one frame is 1/30 seconds and the fluorescence cycle of the fluorescent light is 1/120 seconds, integer multiples (four times) of the fluorescence cycle of the fluorescent light coincide with the cycle of one frame. Accordingly, the timings for starting and ending the integration operation of the photoelectric conversion element connected to each horizontal selection line becomes the same at the nth frame and the following (n+1)th frame. Thus, in the image plane, the brightness of each horizontal line corresponding to a horizontal selection line is fixed for each frame.
The timings for starting and ending the integration operation for different horizontal selection lines in the same frame are all not the same for the fluorescent cycle of the fluorescent light. Thus, a bright and dark horizontal stripe appears for four cycles on an image plane. This phenomenon is flicker noise that appears in an image generated under a fluorescent light having a fluorescent cycle of 1/120 second.
When the cycle of one frame is 1/30 second and the fluorescent cycle of the fluorescent light is 1/100 second, integer multiples of the fluorescence cycle does not coincide with the cycle of one frame. Approximately 3.3 times the fluorescence cycle of the fluorescent light coincides with one frame. Accordingly, the timings for starting and ending the integration operation of the photoelectric conversion element connected to each horizontal selection line are not the same for the nth frame and the following (n+1)th frame. Thus, in the image plane, the brightness of each horizontal line corresponding to a horizontal selection line is not fixed for each frame.
The timings for starting and ending the integration operation for different horizontal selection lines in the same frame are all not the same for the fluorescent cycle of the fluorescent light. Thus, a bright and dark horizontal stripe appears for 3.3 cycles moving upward or downward on the image plane. This phenomenon is flicker noise that appears in an image generated under a fluorescent light having a fluorescent cycle of 1/100 second. Japanese Laid-Open Patent Publication no. 2002-330350 describes the occurrence of the flicker noise.
To prevent flicker noise from being produced under a fluorescent light having a fluorescence cycle of 1/120 seconds, in a range in which the cycle of one frame is 1/30 seconds, the integration time of each photoelectric conversion element is set to an integer multiple of 1/120 seconds, that is, 1/120 seconds, 2/120 seconds, 3/120 seconds, or 4/120 seconds.
However, an integration time that enables the prevention of flicker noise for both of the fluorescence cycles 1/120 seconds and 1/100 seconds does not exist in the frame cycle of 1/30 seconds.
Accordingly, an imaging device incorporating an auto gain control (AGC) circuit, which adjusts the brightness of the generated image, has been proposed. The AGC circuit determines the fluorescence cycle of a fluorescent light and adjusts the integration time of each photoelectric conversion element in accordance with the fluorescence cycle.
FIG. 1 shows an example of an AGC circuit 100, which is provided with a flicker noise cancellation function. The AGC circuit 100 includes an average brightness calculation circuit 1, a brightness ratio calculation circuit 2, a total gain calculation circuit 3, a decoder circuit 4, and a flicker noise detection circuit 5. The average brightness calculation circuit 1 receives brightness data BD from an image sensor block and calculates an average brightness Yave for each frame based on the brightness data BD. The brightness ratio calculation circuit 2 calculates the ratio between the average brightness Yave and a predetermined target brightness Yset. Then, the brightness ratio calculation circuit 2 provides an output signal representing the ratio to the total gain calculation circuit 3.
The total gain calculation circuit 3 holds the total gain of the previous frame and calculates the total gain of the present frame based on the output signal of the brightness ratio calculation circuit 2 and the total gain of the previous frame.
The decoder circuit 4 receives the total gain from the total gain calculation circuit 3 and, based on the total gain, generates a gain adjustment signal A1 and an integration time adjustment signal A2. The gain adjustment signal A1 is used to adjust the gain of an amplifier in the image sensor block. The integration time adjustment signal A2 is used to adjust the integration time for each of the photoelectric conversion elements.
The flicker noise detection circuit 5 detects whether the fluorescent light fluorescence cycle is 1/100 or 1/120 seconds, that is, whether the AC power frequency is 50 Hz or 60 Hz, and provides a corresponding detection signal FC to the decoder circuit 4. Based on the detection signal FC, the decoder circuit 4 generates the integration time adjustment signal A2 for selecting the integration time that prevents flicker noise when the fluorescence cycle is 1/100 seconds or the integration time that prevents flicker noise when the fluorescence cycle is 1/120 seconds.
The flicker noise detection circuit 5 will now be discussed with reference to FIG. 2. The flicker noise detection circuit 5 includes an average brightness calculation circuit 6, a product sum calculation circuit 8, and a comparison circuit 9. The average brightness calculation circuit 6 is provided with the brightness data BD output from the image block sensor. Referring to FIG. 5, the average brightness calculation circuit 6 calculates the average brightness at two predetermined average brightness calculation regions 7a and 7b of the image block sensor for each frame.
The average brightness calculation regions 7a and 7b correspond to two horizontal lines that are separated from each other by a predetermined quantity D of horizontal lines. The quantity D is represented by the following equation:D=V×(1/2−(remainder of A/B))/(A/B)
In the equation, A represents the fluorescent light fluorescence frequency Hz, V represents the total number of horizontal selection lines, and B represents the frame rate fps.
As described in Japanese Laid-Open Patent Publication No. 2002-330350, from the horizontal lines in a frame, the horizontal line having the maximum brightness and the horizontal line having the minimum brightness are selected as the average brightness calculation regions 7a and 7b, which are separated from each other by the quantity D of horizontal lines.
For example, when the fluorescence cycle is 1/100 seconds and the frame rate is 30 fps, the quantity D of the horizontal lines between the average brightness calculation regions 7a and 7b is set to 1/20 of the total number V of horizontal selection lines.
The product sum calculation circuit 8 receives the average brightness Yn of the average brightness calculation regions 7a and 7b calculated by the average brightness calculation circuit 6. Then, the product sum calculation circuit 8 uses the average brightness Yn for each of 15.5 frames to perform product sum calculation and generate a product sum calculation value Yf.
The comparison circuit 9 compares the product sum calculation value Yf with a predetermined threshold value to generate the detection signal FC, which indicates whether the fluorescence cycle is 1/120 or 1/100 when an image is being generated. For example, based on the integration time adjustment signal A2 output from the decoder circuit 4, in a state in which the integration time for preventing flicker noise at a fluorescent cycle of 1/120 is selected, the product sum calculation value Yf is greater than the threshold value when the brightness data BD is generated in a state in which the fluorescence cycle is 1/100 seconds. In a state in which the integration time for preventing flicker noise at a fluorescent cycle of 1/120 is selected, the product sum calculation value Yf is less than the threshold value when the brightness data BD is generated in a state in which the fluorescence cycle is 1/120 seconds.
FIG. 3 shows the product sum calculation circuit 8 in detail. The product sum calculation circuit 8 includes thirty-one stages of shift registers 10 and multipliers 11 and 12. Further, the product sum calculation circuit 8 includes adders 13, 14, and 15. The average brightness Yn of the average brightness calculation regions 7a and 7b for each of the 15.5 frames is sequentially transferred from the first one of the thirty-one series-connected shift registers 10 to the latter ones and stored. Product sum calculation is started when the average brightness Y0 to Y30 for 15.5 frames is stored in the thirty-one stages of shift registers 10. The thirty-one multipliers 11 multiply each average brightness Y0 to Y31 stored in the thirty-one stages of the shift registers 10 with predetermined coefficients cos 0 to cos 30, respectively. The thirty-one multipliers 12 multiply the average brightness Y0 to Y31 stored in the thirty-one stages of the shift registers 10 with predetermined coefficients sin 0 to sin 30, respectively.
The adder 13 receives the output of each multiplier 11 and adds the outputs to calculate the absolute value of the sum. The adder 14 receives the output of each multiplier 12 and adds the outputs to calculate the absolute value of the sum. The adder 15 adds the two absolute values from the adders 13 and 14 and provides the product sum calculation value Yf to the comparison circuit 9.
Referring to FIG. 4, the coefficients cos 0 to cos 30 and sin 0 to sin 30 used by the product sum calculation circuit 8 are values obtained by plotting a cosine wave 16 and a sine wave 17 at intervals of π/3. Each coefficient is determined so that the total sum of the coefficients cos 0 to cos 30 is zero and the total sum of the coefficients sin 0 to sin 30 is zero.
In such a configuration, if the integration time that prevents flicker noise when the fluorescence cycle is 1/120 seconds is selected, flicker noise is not produced and the product sum calculation value Yf decreases to a value close to zero when the fluorescence cycle is 1/120 seconds during the generation of an image.
If the integration time that prevents flicker noise when the fluorescence cycle is 1/120 seconds is selected, flicker noise is produced and the product sum calculation value Yf is increased when the fluorescence cycle is 1/100 seconds during the generation of an image.
Accordingly, the flicker noise detection circuit 5 functions to determine whether an image is being generated in a state in which the fluorescent light fluorescence cycle is 1/100 seconds or 1/120 seconds. During the generation of an image in a state in which the fluorescent light fluorescence cycle is 1/100 seconds, based on the detection signal FC output from the flicker noise detection circuit 5, the decoder circuit 4 selects the integration time that prevents flicker noise from being produced and outputs the integration time adjustment signal A2.